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Abstract 

The control of a multifingered hand slave in order to accurately exert arbitrary forces and 
impart small movements to a grasped object is, at present, a knotty problem in teleoperation. 

Although a number of articulated robotic hands have been proposed in the recent past for 
dexterous manipulation in autonomous robots, the possible use of such hands as slaves in 
teleoperated manipulation is hindered by the present lack of sensors in those hands, and (even if 
those sensors were available) by the inherent difficulty of transmitting to the master operator the 
complex sensations elicited by such sensors at the slave level. 

In this paper an analysis of different problems related to sensor-based telemanipulation is 
presented. The general sensory systems requirements for dexterous slave manipulators are pointed 
out and the description of a practical sensory system set-up for the robotic system we have 
developed is presented. 

The problem of feeding-back to the human master operator stimuli that can be interpreted by 
his central nervous system as originated during real dexterous manipulation is then considered. 
Finally, some preliminary work aimed at developing an instrumented glove designed purposely for 
commanding the master operation and incorporating Kevlar tendons and tension sensors, is 
discussed. 


1. Introduction 

A number of robotic tasks in space will involve operations inside narrow places such as 
small cells, tanks, platforms, or on special extravehicular structures. Some of those tasks will 
require high dexterity and complex sensorymotor control procedures. Environment conditions (e.g. 
zero g) will strongly affect the execution and the performance of the specific task accomplished by 
the robot. Nevertheless, all the other conditions of the particular task could vary according to the 
different procedure used and, in general, the same task will not be repeated in the same conditions. 
For the above reasons, a typical robot for space applications, even if it may possess almost the 
same hardware of a common industrial robot, most often requires remote human control (1). 

We have elected to investigate in this paper a particular, though fundamental, function in 
which teleoperated robots are involved, that is telemanipulation. It is worth observing that 
telemanipulation, which is ultimately aimed at extending the sensing and manipulation capabilities of 
the human operator to the slave robotic system, requires not only a dexterous slave end-effector, but 
also a sensory system able to sense and transmit complex tactile and kinestetic sensations. 
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A number of articulated robotic hands have been proposed in the recent past in the field of 
autonomous robotics for dexterous manipulation (2) (3). The use of such hands as "slaves" in 
teleoperated manipulation is hindered primarily by the present lack of sensors in those hands. 
Furthermore, even if those sensors were available, it would be inherently difficult to convey to the 
operator the complex sensations elicited by such sensors at the slave level. This is a fundamental 
problem in telepresence: the telemanipulation system should allow the human operator not only to 
observe the manipulated objects, but even to feel the physical contact with them. 

Current state of the art in telemanipulated end-effectors includes joysticks and handles, or 
grippers, incorporating some simple sensors. At the master device level, some additional 
sophistication has been achieved with the DataGlove (4), which incorporates fiberoptic position 
sensors located at the finger joints, and a 6-degree-of-freedom tracking device mounted at the wrist 
which provides information about position, orientation and whole configuration of the human 
operator hand in 3D space. 

It is the objective of the research reported here to investigate the design principles and to 
identify the main problems involved in the development of a master-slave system which could be 
used for sensor-based telemicromanipulation experiments. In particular our ultimate goal is to 
render a human operator able to control a multifingered hand slave in a truly dexterous way, that is 
to accurately exert arbitrary forces and impart small movements to a grasped object belonging to a 
remote operational space. 

In the following some basic considerations on the design of a telemanipulation system are 
discussed first. These considerations are related to the general system architecture and to the 
requirements for slave and master devices, as well as to the sensory systems which have to be 
integrated in their structures in order to achieve an active bilateral control of the manipulative 
operation. The following paragraphs deal with the description of the simple robotic system we are 
currently developing in order to investigate some basic issues in telemicromanipulation. The robotic 
system consists of a tendon actuated robotic slave finger with joint rotation and torque sensors and 
tactile sensors at the fingertip, and of an anthropomorphic glove-like exoskeleton incorporating 
actively controlled joints for reproducing kinesthetic sensations on the master human operator. 


2. General design considerations for a telemanipulation robotic system 

As a first step towards the development of a telemanipulation robotic system, we have 
attempted to define some general specifications both on the principles of operation (for example, the 
way in which the whole process could be performed) and on the specific hardware characteristics 
that a master-slave system should possess for carrying out telemicromanipulation procedures. 

As far as the human control of the remote manipulative task is concerned, we assume that 
the operator (either the astronaut or a ground operator) will usually not supervise the operation just 
by giving commands to a computer and leaving the execution of semi-automated manipulation 
procedures to the robotic system. Rather, we assume a direct and continuous human control on the 
operation. In fact we have even conceived a strict isomorphic relation (isomorphism ) between 
the robotic hand and the human master hand. 

The isomorphic assumption leads to a clear emphasis in our approach for the concept of 
telepresence (or tele-existence) of which the telemanipulation task represents only one (although 
fundamental, because it is "active") aspect. For this reason, we have imagined a scenario 
conceptually rather similar to that already introduced in the virtual display-control interface for the 
DataGlove (4), where the human operator, wearing a video display in which the video image of the 
operational space is represented, feels himself as present in the remote working place. A pictorial 
representation of the possible scenario is given in Figure 1. 
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In analogy with the expected performance of the visual feedback in the case of ideal 
telepresence, we assume that also the systems designed for feeding-back the contact information 
detected by the slave robotic hand during manipulation procedures will generate adequate stimuli 
(5) in the human operator hand. The term "adequate stimuli" means that the sensations evoked to the 
human brain cortex when the manipulation procedure is performed directly by the human hand 
should be similar to those evoked in the "artificial" situation in which the manipulaton procedure is 
actually performed by the artificial slave hand. This fact implies, that the contact information (i.e. 
that related to exteroceptors) should be conveied physically to the hand of the master operator, 
without any display interface, such as a computer-graphic display or other equivalent devices. 

Another important consideration for the definition of a robotic system for telemanipulation 
refers to the availability of a dexterous robotic hand equipped with sensory systems of various 
kinds. This requirement originates from the very concept of dexterous manipulation, which 
requires an articulated effector equipped with proprio- and exteroceptive sensors, commanded 
through a hierarchy of sensory-motor control procedures. Only the availability of an appropriate set 
of sensors mounted on an artificial dexterous hand will allow the slave to perform "blind' (e.g. 
without direct visual feedback) tele-commanded explorations. 

From a design point of view, it is important to note that the kinematics of the slave device 
could even be different from the master’s one. The control of the slave in this case would be 
performed by introducing coordinate transformations. In the particular case the human control is 
obtained by using an instrumented glove, also the actuation system could be somewhat different 
from the slave actuation system: in this case a transformation between the master actuator space and 
the slave actuator space is needed. 
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3. Sensory requirements for a dexterous slave system 


As discussed above, a primary need for a telemanipulation system is the presence of sensory 
systems located at the slave hand, and capable of extracting information about the contact conditions 
with the surrounding environment. These sensory systems, which allow the slave to be controlled 
during the execution of complex manipulation procedures, can be classified, according to the 
functional content of the information they extract, as: 

a) teleceptors, which provide information about the remote place as a whole (artificial eyes and 
ears for long range action; proximity sensors for short range action, etc.); 

b) exteroceptors, which detect information on the contact between the robot effector and the 
external environment ( this category includes all the "skin" sensors); 

c) proprioceptors, which sense position, orientation and relative movements of the various 
links of the robot effectors (angular joint rotation and internal force sensors); 

d) enteroceptors, which monitor the functional conditions of the various mechanical and 
electronic components of the slave system. 

That all these sensors are essential for the actual control of the whole telemanipulation 
procedure is easy to perceive by considering, for example, how fundamental is the skilled 
integration of visual and tactile/force sensing modalities for executing even simple manipulation 
procedures. 

We intend to focus here our attention on categories b) and c) because these receptors are 
directly related to the hardware of the slave end-effector . 

In general, although external and internal sensors (as exteroceptors and proprioceptors are 
commonly named in robotics) for space robotic end-effectors are based on the same principles of 
operation and on the same technologies as industrial robots, it must be taken into account that in the 
space environment some requirements on weight and size are critical. 

Contact sensors (external force sensors and tactile sensors) play a very important role, 
among exteroceptors, on the slave hand. The ability to resolve the six components of the resultant 
forces and torques acting on the contact regions of the slave hand leads to a more accurate control of 
manipulative procedures (6). Force/torque resultant sensors can be positioned either at the wrist of 
the robotic hand or/and inside the distal phalanxes of each finger, being resolution improved while 
the sensor moves towards the fingertip. Besides determining contact force and torque, external 
force/torque resultant sensors provide also extremely useful information about the possible slippage 
of the manipulated object. 

Tactile sensing can be considered as complementary to force sensing for the control of 
manipulation procedures. Although tactile sensing has been regarded so far in the field of robotics 
mostly as the artificial sensing modality devoted to determine pressure distribution over the contact 
regions of the end-effector, "tactile" sensing can actually provide a much wider and larger amount of 
information. Several technologies have been used to implement the former approach (6). At present, 
however, not only mechanical but also physical and chemical properties of the contact regions are 
considered as important and useful for perceptual purposes and for fine manipulation. Moreover, a 
"dynamic" approach to the analysis of tactile data is now being stressed, with particular emphasis to 
the control of exploratory and "blind" recognition procedures (7). Real dexterous behavior can result 
from a synthesis of force and tactile sensing. In fact external force measurements can be effectively 
combined with the detection of texture, local shape, roughness and "thermal properties" of the 
manipulated object in order to derive a more detailed description of the object and to more accurately 
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control fine motion and force at the articulated slave hand. 


Proprioceptive sensors have the function of indicating to the controller the relative position 
between the links of the slave hand. The knowledge, at any time, of the "joint vector" allows not 
only to implement pure position control procedure but also, in combination with internal 
force/torque information, the hybrid control of manipulation procedures. 

In order to demonstrate the importance of providing a slave hand with an appropriate set of 
exteroceptive and proprioceptive sensory systems, we have implemented a set of simple exploratory 
procedures by utilizing a tendon actuated, anthropomorphic 4 degree-of-freedom finger equipped 
with joint rotation and torque internal sensors (8). External stimuli deriving from the operational 
space during contact between the finger and the environment are detected by an "epidermal tactile 
sensor positioned at the fingertip. The same epidermal sensor, fabricated with a ferroelectric 
polymer film, could be also useful for detecting dynamic thermal properties of the contact regions. 


4. Considerations on exteroceptive and proprioceptive feedback for the master 
hand. 


Based on the assumption discussed in paragraph 2. of the isomorphic relation between the 
dexterous robotic slave hand and the human master hand, the problem of specifying the 
characteristics of the interfacing system has to be addressed. The functional operations required to 
this interface system are : a) to collect proprioceptive data from the master hand in order to command 
the slave operation , and b) to receive the exteroceptive information deriving from the slave and 
translating them into adequate feedback stimuli to the human hand. 

Functions a) and b) must be performed by sensory and actuating systems positioned in 
contact with the human hand or with a deformable or rigid support wrapping the hand up. A clear 
example of such a structure is the already mentioned DataGlove (4), which incorporates joint 
angular rotation sensors but allows the hand to reach all possible kinematics configurations. 

Manoeuvrability and ergonomic considerations are critical aspects in the design of the master 
telemanipulation system: these requirements are considerably emphasized in the case of 
telemicromanipulation, where the range of fine motion is very critical. For these reasons it is 
unlikely that the whole human master system could significantly differ morphologically and 
functionally from the human hand. 

The system we have devised for the master hand consists of an instrumented glove 
possessing not only position sensors but also an actuating slave-commanded system for finger 
joints. The instrumented glove is depicted in Figure 2. 

Kevlar tendons are routed along the back and the palm of the glove in order to actuate 
directly each phalanx according to a push-pull configuration. Tendon tension sensors, located at the 
wrist level, control the force-reflecting master-slave and slave-back-to-master procedures. Motors 
are also located remotely, in a structure beyond the wrist, in order to allow better hand 
manoeuvrability. An external glove protects the instrumented one and all Kevlar transmission 
tendons. Work is in progress for the realization of a prototype of the tendon-commanded glove. We 
must point out that, although the force-reflecting problem seems theoretically feasible, in practice 
several problems, derived from friction and real time coordination, could arise during the control 
phase. 


An open problem for the realisation of a compact and compliant glove-like master device is 
the definition of the "actuating" or "stimulating" systems aimed at re-creating appropriate 
exteroceptive stimuli on the virtual contact regions of the human hand. A reasonable solution to this 


105 



problem could be the use of local micro-actuators arrays capable of stimulating the human master's 
hand skin, according to a coherent spatio-temporal pattern. Other micro-actuators technologies (e.g. 
solenoid arrays, piezoelectric arrays or micromachined silicon active structures) have not been 
applied yet, owing probably to either volume or compliance constraints. Even feedback-to-master 
procedures for replicating "thermal" sensations could be implemented by available technology, 
should the dimensional vs. manoeuvrability problem find a practical solution. 


ANGULAR 
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Figure 2. Scheme of the instrumented glove for fine telemanipulation 


5. Conclusion 

In this paper we have outlined the general problems of telemanipulation with emphasis on the 
particular case of fine manipulation tasks. In fact, we believe that this domain of applications, 
although extremely challenging, is going to be crucial to any wide diffusion of robotics teleoperation 
technology in space, and even elsewhere. 

The problem of controlling very fine manipulation has to be addressed if, for instance, 
delicate assembly operations or remote handling of delicate samples have to be performed. In this 
class of operations, that will be increasingly important in space missions, a simple gripper will 
certainly not be sufficient, but even a multifingered hand will not be entirely useful if not equipped 
with adequate sensors. 

An important aspect that we pointed out is that, although the use of joint rotation and torque 
sensors and of some contact sensors is an essential requirement for dexterous behavior, very fine 
manipulation requires, in addition, the use of true tactile sensors capable of discriminating very 
small surface indentation at the finger surface. For these operations the control of slippage will also 
be crucial; to this aim, perhaps even a sensitive force/torque fingertip sensor will not suffice, and 
skin-like distributed tactile sensors capable of sensing locally shear stress will be necessary. 

Another important aspect of teleoperation, which to some extent comprises the same 
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functional aspects of the problem, is tele-existence. In this field, sensing the tiny features of contact 
becomes a key part of the process of perceiving fully a remote reality. Measuring local indentation, 
and perceiving texture, thermal properties, compliance and other parameters of the touched object by 
dynamic exploration is a fundamental component of the process by which a human master operator 
can remotely "construct" a mental image of the environment which closely resembles the real one. 

Based on the above considerations, we intend to address in depth in the near future the 
problem of "enriching" telesensations with information other than just vision. Teletactile sensing is a 
fundamental (even if not the only) part of the sensory information necessary to the master in order 
to "generate" a replica of the remote environment as faithful as possible. In this context, particular 
attention will be devoted to investigate issues of psychophysics, inherently associated with 
telesensation. Our approach to the problem of transmitting fine tactile sensations from the slave to 
the master has been outlined here. Further research, now in progress, will address this aspect more 
thoroughly, along with the key problem of using appropriate sensory-motor control techniques to 
extract dynamically tactile data by teleoperated exploratory procedures. 

It may be worth pointing out, in conclusion, that telemicromanipulation can be very useful 
for a number of applications other than space. One of the applications we are investigating is, for 
instance, telemicrosurgery. 
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